System and method providing collaborative interaction

• US 10,270,819 B2
• Filed: 11/26/2014
• Issued: 04/23/2019
• Est. Priority Date: 05/14/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Conklin, Skji Martens, Davidson, Philip Lindsley, Archambault, Anne, Lee, Joo Young, Berg, Jeff, Zuo, Peter, Rogan, Phil
• Primary Examiner(s): Weng, Pei Yong
• Application Number:
  US14/555,458
• Publication Number:
  US 20150334139A1
• Time in Patent Office: 1,609 Days
• CPC Class Codes: G06F 3/03 Arrangements for converting the position or the displacement of a member into a coded form
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks

Abstract

A method for providing collaborative interaction between a receiver computing device and a sender computing device is provided. The method includes at the receiver computing device, establishing a local peer-to-peer wireless connection with a sender computing device, receiving from the sender computing device via the local peer-to-peer wireless connection data for a GUI of a remotely executed application program that is executed on the sender computing device, displaying the GUI on a display associated with the receiver computing device, receiving user input from an input device associated with the receiver computing device indicating a user interaction with the GUI, sending the user input to the sender computing device, receiving from the sender computing device data indicating an updated state of the GUI generated by the sender computing device based on the user input, and displaying the updated state of the GUI on the display associated with the receiver computing device.

Claims

1. A method for providing collaborative interaction between a receiver computing device and a sender computing device, the method comprising:

at the receiver computing device, establishing a local peer-to-peer wireless connection with a sender computing device;

receiving from the sender computing device, via the local peer-to-peer wireless connection, data for a graphical user interface (GUI) of a remotely executed application program that is executed on the sender computing device;

displaying the GUI on a display associated with the receiver computing device;

establishing another local peer-to-peer wireless connection with another sender computing device;

receiving from the other sender computing device, via the other local peer-to-peer wireless connection, data for another GUI; and

following reception of the other GUI from the other sender computing device, while the GUI is displayed on the display, programmatically inhibiting concurrent display of the other GUI on the display.

2. The method of claim 1, wherein the local peer-to-peer wireless connection is a WI-FI Direct connection.

3. The method of claim 1, further comprising, prior to receiving the remotely executed GUI, at the sender computing device:

presenting a device-discovery graphical user interface including a list of a plurality of identified receiver computing devices including the receiver computing device;

receiving a selection input of the receiver computing device from an input device associated with the sender computing device; and

sending the GUI to the receiver computing device in response to receiving the selection input.

4. The method of claim 3, where the plurality of identified receiver computing devices in the list are identified via wireless connectivity.

5. The method of claim 3, where the plurality of identified receiver computing devices in the list are identified based on a physical or logical distance between the sender computing devices and the receiver computing devices.

6. The method of claim 5, where the distances are determined based on one or more of WI-FI communication, Bluetooth communication, near field communication (NFC), and subnet discovery.

7. The method of claim 3, where a relative position of each of the plurality of identified receiver computing devices in the list is determined based on a frequency of bi-directional communication sessions between the receiver computing devices and the sender computing device and the date and time of bi-directional communication sessions between the receiver computing devices and the sender computing device.

8. The method of claim 3, where the list includes preview images associated with the plurality of identified receiver computing devices.

9. The method of claim 3, further comprising at the sender computing device, hiding the device-discovery graphical user interface and displaying a connection bar in response to receiving the selection input of the receiver computing device from an input device associated with the receiver device.

10. The method of claim 1, where the GUI is displayed on the display within a window, the window associated with an application program locally executed on the receiver computing device.

20. The method of claim 1, wherein the display is a touch sensitive display and an input device is a touch sensor of the touch sensitive display configured to sense touch of a digit of a user or a stylus.

21. The method of claim 20, further comprising:

receiving a user input from the input device associated with the receiver computing device indicating user interaction with the GUI, wherein the user input is a touch input corresponding to the sensed touch of the digit of the user or an ink input corresponding to the sensed touch of the stylus;

sending the user input to the sender computing device;

receiving from the sender computing device data indicating an updated state of the GUI generated by the sender computing device based on the user input; and

displaying the updated state of the GUI on the display associated with the receiver computing device.

11. A receiver computing device comprising:

a processor configured to;

establish a local bi-directional peer-to-peer wireless connection with a sender computing device co-located with the receiver computing device;

receive from the sender computing device, via the local bi-directional peer-to-peer wireless, connection data for a graphical user interface (GUI) of a remotely executed application program executed on the sender computing device;

display the GUI on an interactive touch sensitive display associated with the receiver computing device;

establish another local peer-to-peer wireless connection with another sender computing device;

receive from the other sender computing device, via the other local peer-to-peer wireless connection, data for another GUI;

following reception of the other GUI from the other sender computing device, while the GUI is displayed on the display, programmatically inhibit concurrent display of the other GUI on the display.

12. The receiver computing device of claim 11, wherein the processor is further configured to:

receive a plurality of concurrent user inputs from the interactive touch sensitive display indicating a user interaction with the GUI, wherein each user input of the plurality of concurrent user inputs is a touch input corresponding to a sensed touch of a digit of a user or an ink input corresponding to a sensed touch of a stylus;

send the user inputs to the sender computing device;

receive from the sender computing device data indicating an updated state of the GUI generated by the sender computing device based on the user inputs; and

display the updated state of the GUI on the interactive touch sensitive display;

where the updated state reflects a graphical change in the GUI based on a command executed via the application program remotely on the sender computing device.

13. The receiver computing device of claim 11, where the local bi-directional peer-to-peer wireless connection is one of a Bluetooth connection and a WI-FI connection.

14. The receiver computing device of claim 11, where the interactive touch sensitive display includes a capacitive touch sensor configured to sense touch of the digit of the user or the stylus.

15. The receiver computing device of claim 11, wherein the processor is further configured to receive audio data associated with the GUI from the sender computing device associated with the GUI and project the audio data via an audio device in the receiver computing device.

16. The receiver computing device of claim 11, where the receiver computing device display is larger than a display associated with the sender computing device.

17. A method for providing collaborative interaction between a receiver computing device and a sender computing device, the method comprising:

at a receiver application program executed by the receiver computing device, establishing a WI-FI or Bluetooth connection with a sender computing device via a sender application program executed by the sender computing device;

receiving, from the sender application program, data for a graphical user interface (GUI) of a second application program executed on the sender computing device;

displaying the GUI on an interactive touch sensitive display associated with the receiver computing device;

establishing another local peer-to-peer wireless connection with another sender computing device;

receiving from the other sender computing device, via the other local peer-to-peer wireless connection, data for another GUI;

following reception of the other GUI from the other sender computing device, while the GUI is displayed on the display, programmatically inhibiting concurrent display of the other GUI on the display;

receiving a plurality of concurrent user inputs from an interactive touch sensitive display associated with the receiver computing device indicating user interaction with the displayed GUI, wherein each user input of the plurality of concurrent user inputs is a touch input corresponding to a sensed touch of a digit of a user or an ink input corresponding to a sensed touch of a stylus;

sending the user inputs to the sender application program;

receiving from the sender application program data indicating an updated state of the GUI generated by the second application program based on the user inputs; and

displaying the updated state of the GUI on the interactive touch sensitive display.

18. The method of claim 17, where the updated state reflects a graphical change in the GUI based on an action executed via the second application program remotely on the sender computing device.

19. The method of claim 17, further comprising, prior to receiving the remotely executed GUI, at the sender computing device:

presenting a device-discovery graphical user interface including a list of a plurality of identified receiver computing devices including the receiver computing device;

receiving a selection input of the receiver computing device from an input device associated with the sender computing device; and

sending the GUI to the receiver computing device in response to receiving the selection input.