SOCIAL INTERACTION FOR REMOTE COMMUNICATION

• US 20170038829A1
• Filed: 11/30/2015
• Published: 02/09/2017
• Est. Priority Date: 08/07/2015
• Status: Active Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Won, Andrea, Lanier, Jaron, Luraschi, Javier Arturo Porras, Chang, Wayne
• Application Number:
  US14/953,662
• Publication Number:
  US 20170038829A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Techniques for enabling two or more remotely located users to interact with one another and for causing virtual content that corresponds to individual users of the two or more users to augment virtual representations of the individual users in a remote communication environment is described. A service provider can receive image data and tracking data associated with a first user corresponding to a first device. Further, a service provider can cause a virtual representation of the first user to be presented on a display of a second device corresponding to a second user, determine an interaction between an object associated with the second user and the virtual representation of the first user, and based at least in part on determining the interaction, cause virtual content to be presented on the virtual representation of the first user on at least the display.

Claims

1. A system comprising:

one or more processors;

memory; and

one or more modules stored in the memory and executable by the one or more processors to perform operations comprising;

determining initiation of a communication between a first device associated with a first user and a second device associated with a second user, the second device being remotely located from the first device;

receiving, from an image capturing device associated with the first device, image data associated with the first user;

receiving, from a tracking device associated with the first device, tracking data associated with the first user;

causing, based at least in part on the image data, a virtual representation of the first user to be presented on a first display corresponding to the second device;

determining an interaction between an object associated with the second user and the virtual representation of the first user;

causing virtual content to be presented on at least the first display corresponding to the second device in a position on the virtual representation of the first user corresponding to the interaction; and

causing, based at least in part on the tracking data, the virtual content to track with movement of the first user.

2. The system as claim 1 recites, wherein:

the first display comprises a touchscreen display; and

the interaction is between the object and a portion of the touchscreen display corresponding to the virtual representation.

3. The system as claim 2 recites, wherein the object comprises a body part of the second user.

4. The system as claim 1 recites, wherein the object comprises an input peripheral device controlled by the second user.

5. The system as claim 1 recites, the operations further comprising, based at least in part on the interaction, causing the virtual content to be presented on a second display associated with the first device in the position on the virtual representation of the first user.

6. The system as claim 1 recites, the operations further comprising determining a first unique identifier associated with the first user and second unique identifier associated with the second user.

7. The system as claim 6 recites, the operations further comprising mapping the virtual content to at least one of the first unique identifier or the second unique identifier.

8. The system as claim 6 recites, wherein permissions data associated with at least one of the first unique identifier or the second unique identifier indicates authorizations associated with at least one of the first user or the second user for terminating the virtual content from being presented on at least the first display.

9. The system as claim 1 recites, the operations further comprising terminating the virtual content from being presented on at least the first display based at least in part on expiration data associated with the virtual content.

10. A method for causing virtual content to be presented in a remote communication environment, the method comprising:

receiving, from an image capturing device associated with a first device, image data associated with a first user corresponding to the first device;

causing, based at least in part on the image data, a virtual representation of the first user to be presented on a second device corresponding to a second user;

determining an interaction between an object associated with the second user and the virtual representation of the first user; and

based at least in part on the interaction, causing virtual content to be presented on the virtual representation of the first user on a first display of the first device and a second display of the second device.

11. The method as claim 10 recites, wherein causing the virtual content to be presented on the virtual representation of the first user comprises causing the virtual content to be rendered in a position on the virtual representation of the first user corresponding to the interaction.

12. The method as claim 11 recites, further comprising:

receiving, from a tracking device associated with the first device, tracking data associated with the first user; and

causing, based at least in part on the tracking data, the virtual content to persist in the position on the virtual representation of the first user such to track with movement of the first user.

13. The method as claim 12 recites, wherein the image data and the tracking data are received over a period of time.

14. The method as claim 10 recites, further comprising prior to causing the virtual content to be presented on the virtual representation of the first user on the first display and the second display, accessing first permissions data associated with the first user and second permissions data associated with the second user.

15. The method as claim 14 recites, further comprising, based at least in part on accessing the first permissions data and the second permissions data, determining that the interaction is authorized between the first user and the second user.

16. The method as claim 14 recites, further comprising:

determining that the remote communication environment includes the first user, the second user, and a third user;

accessing third permissions data associated with the third user; and

determining, based at least in part on at least one of the first permissions data, the second permissions data, or the third permissions data, that the third user is not authorized to view the virtual content.

17. The method as claim 14 recites, further comprising:

terminating a first communication associated with causing the virtual content to be presented on the virtual representation of the first user on the first display and the second display;

determining initiation of a new communication between the first user device and the second user device;

determining that the virtual content is mapped to the first unique identifier and the second unique identifier;

determining that the virtual content has yet to expire; and

causing the virtual content to be presented on the virtual representation of the first user on the first display and the second display for at least a portion of the new communication.

18. One or more computer storage media having computer-executable instructions that, when executed by one or more processors, configure the one or more processors to perform operations comprising:

receiving, from an image capturing device associated with a first device, image data associated with a first user corresponding to the first device;

receiving, from a tracking device associated with the first device, tracking data associated with the first user;

causing, based at least in part on the image data, a virtual representation of the first user to be presented on a display of a second device corresponding to a second user;

determining an interaction between an object associated with the second user and the virtual representation of the first user; and

based at least in part on the interaction, causing virtual content to be presented on the virtual representation of the first user on at least the display, wherein the virtual content is positioned on the virtual representation of the first user based on the tracking data and to visually indicate a position on the virtual representation of the first user where the object interacts with the first user.

19. One or more computer storage media as claim 18 recites, wherein causing the virtual content to be presented on the virtual representation of the first user comprises causing, based at least in part on the tracking data, the virtual content to persist in the position on the virtual representation of the first user such to track with movement of the first user.

20. One or more computer storage media as claim 18 recites, wherein the virtual content corresponding to the interaction is defined by the second user.