APPLICATION SHARING

• US 20100257450A1
• Filed: 04/03/2009
• Published: 10/07/2010
• Est. Priority Date: 04/03/2009
• Status: Active Grant

• Current Assignee: Sococo, Inc.
• Original Assignee: Social Communications Company
• Inventors: Go, Alex Say, Petter, Viadimir
• Granted Patent: US 8,407,605 B2
• Time in Patent Office: Days
• US Class Current: 715/733
• CPC Class Codes: G06F 2209/545 Gui
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Ones of the windows associated with a software process are identified in a screen layout on a local display of a sharer network node. On the sharer network node, an image of the identified windows as they are arranged in the screen layout and free of obscuration by any other windows in the screen layout is generated. The image is transmitted from the sharer network node to a viewer network node. Also, commands derived from input device events on sharer and viewer network nodes are received. The received commands are processed into a command sequence, which is passed to a shared process executing on the sharer network node. Windows associated with the shared process are presented on the sharer network node in accordance with the received command sequence. An image of the presented windows is generated and transmitted from the sharer network node to the viewer network node.

Claims

1. A method, comprising:

in a screen layout on a local display of a sharer network node, identifying ones of the windows that are associated with a software process;

on the sharer network node, generating a composite image of the identified windows as they are arranged in the screen layout and free of obscuration by any other windows in the screen layout; and

transmitting the composite image from the sharer network node to a viewer network node.

2. The method of claim 1, wherein the identifying comprises identifying all of the windows in the screen layout that are associated with the software process.

3. The method of claim 1, wherein the identifying comprises identifying ones of the windows in the screen layout that match a handle that is assigned to the software process.

4. The method of claim 3, wherein the identifying comprises identifying a parent window and at least one associated child window that are created by the software process.

5. The method of claim 1, wherein the generating comprises:

determining a layering order of the identified windows in relation to each other that corresponds to the relative layering order of the identified windows in the screen layout;

for each of the identified windows, retrieving a respective image of the window; and

compositing the retrieved images into the composite image in accordance with the determined layering order.

6. The method of claim 5, wherein the determining comprises

for each of the windows in the screen layout, generating a z-order list that associates a respective z-order value with a respective window handle of the window, and

deriving the layering order of the identified windows from the z-order list.

7. The method of claim 6, wherein the deriving comprises:

for each of the z-order values in the z-order list, matching the associated window handle with the window handle of a respective one of the identified windows; and

ordering the identified windows in the layering order in accordance with the respective z-order values in the z-order list associated with the ones of the z-order values determined to match the window handles of the identified windows.

8. The method of claim 5, wherein the generating additionally comprises determining two-dimensional locations of the identified windows in the screen layout.

9. The method of claim 1, wherein:

the identifying comprises identifying ones of the windows that are associated with a designed group of software processes; and

the generating comprises

determining a layering order of the identified windows in relation to each other that corresponds to the relative layering order of the identified windows in the screen layout

for each of the identified windows, retrieving a respective image of the window, and

compositing the retrieved images into the composite image in accordance with the determined layering order

10. The method of claim 1, wherein the generating comprises generating the composite image of the identified windows that includes any contents of the identified windows that are located outside a visible desktop window that contains the screen layout.

11. The method of claim 1, wherein a respective image of each of the identified windows is stored in a respective memory buffer, and the generating comprises retrieving each of the images from the respective memory buffers and compositing the retrieved images into the composite image.

12. The method of claim 1, further comprising setting the windows that are associated with the software application to a layered windows style such that a respective image of each of the identified windows is stored in a respective memory buffer.

13. The method of claim 1, wherein the transmitting is performed in response to a request from the viewer network node to view screen data associated with the software process.

14. The method of claim 13, further comprising transmitting the composite image from the sharer network node to one or more other viewer network nodes in response to a respective request from each of the other the viewer network nodes to view screen data associated with the software process, wherein the composite image is transmitted contemporaneously to each of the remote network nodes.

15. The method of claim 1, wherein the identifying, the generating, and the transmitting are performed in a context defined by a virtual area in which a sharer at the sharer network node and a viewer at the viewer network node both have presence.

16. The method of claim 15, further comprising:

on the local display, displaying a spatial layout of zones of the virtual area;

on the local display, presenting navigation controls and interaction controls, wherein the navigation controls enable the sharer to specify where to establish a presence in the virtual area and the interaction controls enable the sharer to manage interactions with the viewer in the virtual area;

establishing a respective presence of the sharer in each of one or more of the zones in response to input received via the navigation controls; and

on the local display, depicting respective graphical representations of the sharer and the viewer in each of the zones where the sharer and the viewer respectively have presence.

17. The method of claim 16, wherein the identified windows contain contents of at least one data file that is associated with the software process, and further comprising:

in response to input by the sharer, associating the data file with a viewscreen object in a respective one of the zones; and

performing the identifying, the generating, and the transmitting in response to a request from the viewer network node to view the data file on the viewscreen object.

18. The method of claim 17, further comprising presenting the composite image on the viewscreen object.

19. The method of claim 17, further comprising performing the identifying, the generating, and the transmitting on multiple network nodes, and presenting the respective composite images on different respective viewscreen objects in the virtual area.

20. The method of claim 19, further comprising

for each of the viewscreen objects on which a respective one of the composite images is presented, showing the graphical depiction of a respective sharer operating the respective network node from which the respective composite image is transmitted.

21. The method of claim 1, further comprising

sending a request to share screen data associated with a software process executing on the viewer network node;

and

at the sharer network node, receiving a respective composite image of shared screen data from the viewer network node, wherein the receiving and the transmitting are performed contemporaneously.

22. At least one computer-readable medium having computer-readable program code embodied therein, the computer-readable program code adapted to be executed by a computer to implement a method comprising:

in a screen layout on a local display of a sharer network node, identifying ones of the windows that are associated with a software process;

on the sharer network node, generating a composite image of the identified windows as they are arranged in the screen layout and free of obscuration by any other windows in the screen layout; and

transmitting the composite image from the sharer network node to a viewer network node.

23. Apparatus, comprising:

a local display;

a computer-readable medium storing computer-readable instructions; and

a data processing unit coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising

in a screen layout on the local display, identifying ones of the windows that are associated with a software process;

generating a composite image of the identified windows as they are arranged in the screen layout and free of obscuration by any other windows in the screen layout; and

transmitting the composite image to a remote viewer network node.

24. A method, comprising:

receiving locally-generated commands that are derived from local input device events on a sharer network node;

receiving remotely-generated commands that are derived from remote input device events on a remote viewer network node;

processing the received commands into a command sequence;

passing the command sequence to a shared process executing on the sharer network node;

in a screen layout on a local display of the sharer network node, presenting one or more windows that are associated with the shared process in accordance with the received command sequence;

generating an image of the one or more windows as they are presented in the screen layout; and

transmitting the image from the sharer network node to the viewer network node.

25. The method of claim 24, wherein the processing comprises remapping screen location parameter values in the remotely-generated commands from a coordinate system of a shared application image received from the sharer network node to a coordinate system of a main window on the local display.

26. The method of claim 25, wherein the processing comprises arranging into the command sequence (i) the remotely-generated commands with the remapped screen location parameter values and (ii) the locally-generated commands.

27. The method of claim 26, wherein the arranging comprises ordering (i) the remotely-generated commands with the remapped screen location parameter values and (ii) the locally-generated commands based on respective times that the locally-generated commands are derived from the local input device events and respective times that the remotely-generated commands are received.

28. The method of claim 25, further comprising remapping screen location parameter values in the commands derived from the remote input device events from a coordinate system of a main window on a remote display of the viewer network node to a coordinate system of the shared application image.

29. The method of claim 28, wherein the remapping is performed on the viewer network node, and further comprising transmitting the remotely-generated commands with the remapped screen location parameter values to the sharer network node.

30. At least one computer-readable medium having computer-readable program code embodied therein, the computer-readable program code adapted to be executed by a computer to implement a method comprising:

receiving locally-generated commands that are derived from local input device events on a sharer network node;

receiving remotely-generated commands that are derived from remote input device events on a remote viewer network node;

processing the received commands into a command sequence;

passing the command sequence to a shared process executing on the sharer network node;

in a screen layout on a local display of the sharer network node, presenting one or more windows that are associated with the shared process in accordance with the received command sequence;

generating an image of the one or more windows as they are presented in the screen layout; and

transmitting the image from the sharer network node to the viewer network node.

31. Apparatus, comprising:

a local display;

a computer-readable medium storing computer-readable instructions; and

a data processing unit coupled to the memory, operable to execute the instructions, and based at least in part on the execution of the instructions operable to perform operations comprising

receiving locally-generated commands that are derived from local input device events that are associated with the local display;

receiving remotely-generated commands that are derived from remote input device events on a remote viewer network node;

processing the received commands into a command sequence;

passing the command sequence to a shared process being executed by the data processing unit;

in a screen layout on the local display, presenting one or more windows that are associated with the shared process in accordance with the received command sequence;

generating an image of the one or more windows as they are presented in the screen layout and

transmitting the image to the viewer network node.