Integrated interactive space

• US 9,584,766 B2
• Filed: 06/03/2015
• Issued: 02/28/2017
• Est. Priority Date: 02/06/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Pradeep, Vivek, Latta, Stephen G., Bathiche, Steven Nabil, Geisner, Kevin, Brush, Alice Jane Bernheim
• Primary Examiner(s): Anwah, Olisa
• Application Number:
  US14/730,061
• Publication Number:
  US 20150271449A1
• Time in Patent Office: 636 Days
• Field of Search: 348/14.01, 348/14.03, 348/14.07, 348/14.08
• US Class Current: 1/1
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Techniques for implementing an integrative interactive space are described. In implementations, video cameras that are positioned to capture video at different locations are synchronized such that aspects of the different locations can be used to generate an integrated interactive space. The integrated interactive space can enable users at the different locations to interact, such as via video interaction, audio interaction, and so on. In at least some embodiments, techniques can be implemented to adjust an image of a participant during a video session such that the participant appears to maintain eye contact with other video session participants at other locations. Techniques can also be implemented to provide a virtual shared space that can enable users to interact with the space, and can also enable users to interact with one another and/or objects that are displayed in the virtual shared space.

Claims

1. A computer-implemented method comprising:

synchronizing a first camera in a first physical environment and a second camera in a second physical environment to generate a simulated three-dimensional shared space shared between a first user in the first physical environment and a second user in the second physical environment;

capturing via the first camera an interaction by the first user with a physical object in the first physical environment to create a drawing in the first physical environment; and

presenting a virtual object representation of the physical object and a representation of the interaction with the physical object in the simulated three-dimensional shared space.

2. A computer-implemented method as described in claim 1, further comprising capturing via the second camera an interaction by the second user with the virtual object in the simulated three-dimensional shared space.

3. A computer-implemented method as described in claim 1, further comprising enabling the first user and the second user to hand the virtual object to one another within the simulated three-dimensional shared space and during a single interactive session.

4. A computer-implemented method as described in claim 1, further comprising enabling the second user to rotate, resize, disassemble, reassemble, or augment the virtual object or the drawing.

5. A computer-implemented method as described in claim 1, wherein the virtual object is a three-dimensional graphical rendering that corresponds to the physical object.

6. A computer-implemented method as described in claim 1, wherein the virtual object is an image of the physical object captured by an optical scanning device.

7. A computer-implemented method as described in claim 1, wherein presenting the representation of the interaction with the physical object comprises presenting a graphical representation of a hand of the first user used in the interaction with the physical object.

8. A system comprising:

one or more computer-readable storage media storing executable instructions; and

one or more processors configured to execute instructions to cause the system to perform operations comprising;

receiving data captured via a first camera, the data captured by the first camera describing an interaction by a first user with a physical object in a first physical environment;

synchronizing the data received from the first camera in the first physical environment and data received from a second camera in a second physical environment to generate a simulated three-dimensional shared space shared between the first user in the first physical environment and a second user in the second physical environment;

generating a virtual object representation of the physical object and a representation of the interaction with the physical object in the simulated three-dimensional shared space; and

receiving subsequent data captured via the second camera, the subsequent data captured by the second camera describing a subsequent interaction by the second user with the virtual object in the simulated three-dimensional shared space, the subsequent interaction by the second user comprising drawing in the simulated three-dimensional shared space.

9. A system as described in claim 8, the operations further comprising receiving subsequent data captured via the second camera, the data captured by the second camera describing an interaction by the second user with the virtual object in the simulated three-dimensional shared space.

10. A system as described in claim 8, the operations further comprising enabling the first user and the second user to hand the virtual object to one another within the simulated three-dimensional shared space and during a single interactive session.

11. A system as described in claim 8, the operations further comprising enabling the second user to rotate, resize, disassemble, reassemble, or augment the virtual object.

12. A system as described in claim 8, wherein the virtual object is a three-dimensional graphical rendering that corresponds to the physical object.

13. A system as described in claim 8, wherein generating the representation of the interaction with the physical object comprises generating a graphical representation of a hand of the first user that was used in the interaction with the physical object in the first environment.

14. A system comprising:

a first camera in a first physical environment;

a second camera in a second physical environment;

a memory and processor system configured to execute a synchronization module implemented to synchronize the first camera in the first physical environment and the second camera in the second physical environment to generate a simulated three-dimensional shared space shared between a first user in the first physical environment and a second user in the second physical environment;

the first camera configured to capture an interaction by the first user with a physical object in the first physical environment to create a drawing in the first physical environment; and

the memory and processor system configured to execute a visualization module implemented to present a virtual object representation of the physical object and a representation of the interaction with the physical object in the simulated three-dimensional shared space.

15. A system as described in claim 14, wherein the first physical environment includes one of a whiteboard, a chalkboard, a painted wall, or a display screen that accepts user input.

16. A system as described in claim 14, further comprising:

the second camera configured to capture an interaction by the second user with the drawing created by the first user; and

wherein the visualization module is implemented to present a virtual representation of the interaction by the second user in the simulated three-dimensional shared space.

17. A system as described in claim 14, wherein the visualization module is implemented to present a virtual representation of the first user performing the interaction.

18. A system as described in claim 14, further comprising:

the second camera configured to capture an interaction by the second user in the second physical environment; and

wherein the visualization module is implemented to present a virtual representation of the interaction by the second user in the simulated three-dimensional shared space.

19. A system as described in claim 14, wherein the visualization module is implemented to present simultaneously while the first camera is configured to capture.

20. A system as described in claim 14, wherein the representation of the interaction with the physical object comprises a graphical representation of a hand of the first user that was used in the interaction with the physical object in the first physical environment.