Projecting a Virtual Copy of a Remote Object

• US 20160267642A1
• Filed: 03/12/2015
• Published: 09/15/2016
• Est. Priority Date: 03/12/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Pejsa, Tomislav, Kantor, Julian, Wilson, Andrew, Benko, Hrvoje, Ofek, Eyal
• Granted Patent: US 10,216,982 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Various systems and methods for projecting a remote object are described herein. In one example, a method includes collecting environment data corresponding to a local environment in which a system is located and detecting a remote object corresponding to a remote user in a remote environment. The method can also include detecting a viewpoint of a local user in the local environment, and projecting the remote object corresponding to the remote user in the local environment based on the viewpoint of the local user, the virtual copy of the remote object to be positioned in the local environment by taking into account geometry of local objects in the local environment.

Claims

1. A system for projecting a virtual copy of a remote object comprising:

a first module to calibrate the system, the calibration comprising collecting environment data corresponding to a local environment in which the system is located;

a second module to receive the virtual copy of a remote object corresponding to a remote user in a remote environment;

a third module to detect a viewpoint of a local user in the local environment; and

a fourth module to project the virtual copy of the remote object in the local environment based on the viewpoint of the local user, the virtual copy of the remote object to be positioned in the local environment by taking into account geometry of local objects in the local environment.

2. The system of claim 1, wherein the third module is to identify locations in the local environment on which the virtual copy of the remote object can be projected.

3. The system of claim 2, wherein the third module identifies the locations based on surface attributes.

4. The system of claim 1, wherein the second module detects that the remote user is in a static position.

5. The system of claim 1, wherein the fourth module rotates and positions the virtual copy based on the viewpoint of the local user and wherein the virtual copy comprises a life size image of the remote user.

6. The system of claim 1 comprising a fifth module to send the local user as a separate virtual copy for projection in the remote environment.

7. The system of claim 1, the remote object comprising depth data, color data, and three dimensional data corresponding to the remote user.

8. The system of claim 1, wherein the fourth module is to:

generate mesh data of the remote object;

detect a surface to project the virtual copy of the remote object in the local environment; and

modify the mesh data based on the environment data.

9. The system of claim 8, wherein the state of the system comprises latency data or retargeting data.

10. The system of claim 1 comprising a sixth module that is to:

detect a conflict with a location in the local environment in which the virtual copy is projected; and

move the location of the projection of the virtual copy of the remote object in the local environment to a retargeted location in the local environment.

11. The system of claim 1, wherein the fourth module is to apply effects to the virtual copy, the effects comprising a non-photorealistic effect.

12. A method for projecting a virtual copy of a remote object comprising:

collecting environment data corresponding to a local environment in which a system is located;

detecting a remote object corresponding to a remote user in a remote environment;

detecting a viewpoint of a local user in the local environment; and

projecting a virtual copy of the remote object corresponding to the remote user in the local environment based on the viewpoint of the local user, the virtual copy of the remote object to be positioned in the local environment by taking into account geometry of local objects in the local environment.

13. The method of claim 12, comprising identifying locations in the local environment on which the remote object can be projected.

14. The method of claim 13, wherein the locations are based on surface attributes.

15. The method of claim 12, comprising applying effects to the virtual copy, the effects comprising saturating the virtual copy and adding a blue tint, applying rim lighting of the virtual copy, and adding a scan line to the virtual copy.

16. The method of claim 12, comprising rotating the virtual copy based on the viewpoint of the local user.

17. The method of claim 12, comprising:

detecting a conflict with a location in the local environment in which the virtual copy of the remote object is projected; and

moving the location of the projection of the virtual copy of the remote object in the local environment to a retargeted location in the local environment.

18. One or more computer-readable storage media for projecting a virtual copy of a remote object comprising a plurality of instructions that, when executed by a processor, cause the processor to:

collect environment data corresponding to a local environment in which a system is located;

detect a remote object corresponding to a remote user in a remote environment;

detect a viewpoint of a local user in the local environment; and

project the remote object corresponding to the remote user in the local environment based on the viewpoint of the local user, the virtual copy of the remote object to be positioned in the local environment by taking into account geometry of local objects in the local environment.

19. The one or more computer-readable storage media of claim 18, wherein the plurality of instructions, when executed by the processor, cause the processor to apply effects to the virtual copy, the effects comprising saturating the virtual copy and adding a blue tint, applying rim lighting of the virtual copy, and adding a scan line to the virtual copy.

20. The one or more computer-readable storage media of claim 18, wherein the plurality of instructions, when executed by the processor, cause the processor to:

detect a conflict with a location in the local environment in which the virtual copy is projected; and

move the location of the projection of the virtual copy in the local environment to a retargeted location in the local environment.