INTERACTIVE VIRTUAL DISPLAY SYSTEM FOR UBIQUITOUS DEVICES
First Claim
1. A method for providing virtual displays of a plurality of remote heterogeneous computing devices on a local display surface, comprising steps for:
- running an instance of a host application on each of a plurality of remote heterogeneous computing devices;
running an agent application on a local display interface module that is in communication with a local display surface;
establishing network connections between each host instance and the agent via one or more networks;
concurrently providing a composite display comprising a separate virtual display corresponding to each of the remote heterogeneous computing devices on the local display surface, each virtual display being generated by the agent from compressed screen images that are periodically transmitted by each corresponding host to the agent via the one or more networks;
providing a local user input mechanism in communication with the display interface module;
wherein the local user input mechanism allows the user to provide user input for interacting with any of the virtual displays corresponding to any of the remote heterogeneous computing devices;
wherein the agent monitors user interaction with any of the virtual displays and sends corresponding control events to corresponding hosts via the one or more networks; and
wherein each host receiving control events interprets those control events to provide input to the corresponding heterogeneous computing device in order to control the actions of the corresponding heterogeneous computing device.
2 Assignments
0 Petitions
Accused Products
Abstract
An “Interactive Virtual Display,” as described herein, provides various systems and techniques that facilitate ubiquitous user interaction with both local and remote heterogeneous computing devices. More specifically, the Interactive Virtual Display uses various combinations of small-size programmable hardware and portable or wearable sensors to enable any display surface (e.g., computer display devices, televisions, projected images/video from projection devices, etc.) to act as a thin client for users to interact with a plurality heterogeneous computing devices regardless of where those devices are located relative to the user. The Interactive Virtual Display provides a flexible system architecture that enables communication and collaboration between a plurality of both local and remote heterogeneous computing devices. This communication and collaboration enables a variety of techniques, such as adaptive screen compression, user interface virtualization, real-time gesture detection to improve system performance and overall user experience, etc.
200 Citations
20 Claims
-
1. A method for providing virtual displays of a plurality of remote heterogeneous computing devices on a local display surface, comprising steps for:
-
running an instance of a host application on each of a plurality of remote heterogeneous computing devices; running an agent application on a local display interface module that is in communication with a local display surface; establishing network connections between each host instance and the agent via one or more networks; concurrently providing a composite display comprising a separate virtual display corresponding to each of the remote heterogeneous computing devices on the local display surface, each virtual display being generated by the agent from compressed screen images that are periodically transmitted by each corresponding host to the agent via the one or more networks; providing a local user input mechanism in communication with the display interface module; wherein the local user input mechanism allows the user to provide user input for interacting with any of the virtual displays corresponding to any of the remote heterogeneous computing devices; wherein the agent monitors user interaction with any of the virtual displays and sends corresponding control events to corresponding hosts via the one or more networks; and wherein each host receiving control events interprets those control events to provide input to the corresponding heterogeneous computing device in order to control the actions of the corresponding heterogeneous computing device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A system for adapting an arbitrary display surface into a thin client having a plurality of concurrent virtual interactive displays each corresponding to one of a plurality of remote heterogeneous computing devices, comprising:
-
a display interface module having one or more communication mechanisms for providing a video input to an arbitrary display surface; wherein the display interface module further includes one or more communication mechanisms for receiving a local user input; wherein the display interface module further includes one or more communication mechanisms for communicating with a plurality of heterogeneous computing devices across one or more networks; wherein the display interface module periodically receives a virtual display from each of the remote heterogeneous computing devices, and wherein the display interface module composes each of the received virtual displays into a combined video output that is provided to the arbitrary display surface via the one or more communication mechanisms for providing the video input to the arbitrary display surface; wherein the display interface module monitors received user input to determine which of the virtual displays comprising the combined video output the user intends to interact with, and sends corresponding control events to the corresponding heterogeneous computing devices via the one or more networks; and wherein each corresponding heterogeneous computing device receiving control events interprets those control events as a local user input to control the actions of the corresponding heterogeneous computing device. - View Dependent Claims (11, 12, 13, 14, 15)
-
-
16. A method for transforming an arbitrary display surface into a thin client for concurrently hosting a plurality of virtual displays corresponding to a plurality of remote heterogeneous computing devices, comprising:
-
a display interface module comprising;
a video output mechanism in communication with a video input mechanism of arbitrary display surface, one or more communication mechanisms for receiving a local user input, and an agent application running on the display interface module in communication with a plurality of heterogeneous computing devices across one or more networks via an instance of a host application running on each of the remote heterogeneous computing devices;wherein the agent composes a composite display from a separate virtual display corresponding to each of the remote heterogeneous computing devices, said composite display being sent by the display interface module to the arbitrary display surface; wherein each virtual display is generated by the agent from compressed screen images that are periodically transmitted by each corresponding host to the agent via the one or more networks; wherein the agent monitors received user input to determine which of the virtual displays comprising the composite display the user intends to interact with, and sends corresponding control events to the corresponding hosts via the one or more networks; and wherein each corresponding host receiving control events interprets those control events as a local user input to control the actions of the corresponding heterogeneous computing device. - View Dependent Claims (17, 18, 19, 20)
-
Specification