AUGMENTED REALITY COMPUTING ENVIRONMENTS - MOBILE DEVICE JOIN AND LOAD

• US 20190310757A1
• Filed: 04/03/2019
• Published: 10/10/2019
• Est. Priority Date: 04/09/2018
• Status: Active Grant

• Current Assignee: Spatial Systems Inc.
• Original Assignee: Spatial Systems Inc.
• Inventors: LEE, Jinha, Agarawala, Anand, Ng, Peter, Hatch, Tyler
• Granted Patent: US 11,086,474 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Disclosed herein are system, method, and computer program product embodiments for providing a local scene recreation of an augmented reality meeting space to a mobile device, laptop computer, or other computing device. By decoupling the augmented reality meeting space from virtual reality headsets, the user-base expands to include users that could otherwise not participate in the collaborative augmented reality meeting spaces. Users participating on mobile devices and laptops may choose between multiple modes of interaction including an auto-switch view and manual views as well as interacting with the augmented reality meeting space by installing an augmented reality toolkit. Users may deploy and interact with various forms of avatars representing other users in the augmented reality meeting space.

Claims

1. A computer-implemented method, comprising:

determining a set of users participating in an augmented reality (AR) meeting space, wherein a user in the set of users is associated with three-dimensional position information that describes a location of the user within the AR meeting space;

determining a set of digital canvases in the AR meeting space, wherein a digital canvas in the set of digital canvases displays digital objects in the AR meeting space and wherein the digital canvas is associated with three-dimensional position information that describes a location of the digital canvas within the AR meeting space;

assembling the set of users and the set of digital canvases into a local scene recreation,

wherein the local scene recreation places the set of users and the set of digital canvases in appropriate locations in the AR meeting space based on the three-dimensional position information, and

wherein the set of users are represented by avatars in the local scene recreation; and

causing the local scene recreation of the AR meeting space to display on a computing device.

2. The method of claim 1, the causing further comprising:

building the local scene recreation in an auto-switch view,

wherein the auto-switch view focuses the display on a point of interest in the AR meeting space; and

adjusting the point of interest in response to a speaking user in the set of users.

3. The method of claim 1, the causing further comprising:

building the local scene recreation in an auto-switch view,

wherein the auto-switch view focuses the display on a point of interest in the AR meeting space; and

adjusting the point of interest in response to an engaged digital canvas in the set of digital canvases.

4. The method of claim 1, further comprising:

determining that the computing device is configured to include an AR kit;

displaying the local scene recreation from a fixed perspective within the AR meeting space;

receiving a gesture interaction from the computing device via the AR kit; and

updating the local scene recreation based on the gesture interaction.

5. The method of claim 1, further comprising:

receiving a manual input from the computing device; and

adjusting the local scene recreation based on the manual input.

6. The method of claim 5, further comprising:

receiving a selected avatar from among the avatars; and

rendering the local scene recreation from a perspective of the user associated with the selected avatar.

7. The method of claim 5, further comprising:

receiving a selected digital canvas from among the digital canvases; and

providing an enhanced view of the selected digital canvas as the point of interest for the local scene recreation.

8. The method of claim 5, further comprising:

determining sticky note properties based on the manual input; and

adding a sticky note to a digital canvas in the AR meeting space.

9. The method of claim 5, further comprising:

determining a photograph to upload based on the manual input; and

adding the photograph to a digital canvas in the AR meeting space.

10. The method of claim 1, wherein the avatars are stored images representing the set of users in the AR meeting space.

11. The method of claim 1, wherein the avatars are stored video feeds captured by a webcam representing the set of users in the AR meeting space.

12. The method of claim 1, wherein the avatars are real-time face captures rendered into a digital video feed to represent the set of users in the AR meeting space.

13. The method of claim 1, further comprising:

receiving a mode selection from the computing device;

displaying the local scene recreation in an auto-switch view or a manual view based on the mode selection.

14. The method of claim 1, wherein the computing device joining the AR meeting space is a mobile device.

15. A system, comprising:

a memory; and

at least one processor coupled to the memory and configured to;

determine a set of users participating in an augmented reality (AR) meeting space, wherein a user in the set of users is associated with three-dimensional position information that describes a location of the user within the AR meeting space;

determine a set of digital canvases in the AR meeting space, wherein a digital canvas in the set of digital canvases displays digital objects in the AR meeting space and wherein the digital canvas is associated with three-dimensional position information that describes a location of the digital canvas within the AR meeting space;

assemble the set of users and the set of digital canvases into a local scene recreation,

wherein the local scene recreation places the set of users and the set of digital canvases in appropriate locations in the AR meeting space based on the three-dimensional position information, and

wherein the set of users are represented by avatars in the local scene recreation; and

cause the local scene recreation of the AR meeting space to display on a computing device.

16. The system of claim 15, wherein the avatars are stored images representing the set of users in the AR meeting space.

17. The system of claim 15, wherein the avatars are stored video feeds captured by a webcam representing the set of users in the AR meeting space.

18. The system of claim 15, wherein the avatars are real-time face captures rendered into a digital video feed to represent the set of users in the AR meeting space.

19. The system of claim 15, the at least one processor further configured to:

receive a mode selection from the computing device; and

display the local scene recreation in an auto-switch view or a manual view based on the mode selection.

20. A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

determining a set of users participating in an augmented reality (AR) meeting space, wherein a user in the set of users is associated with three-dimensional position information that describes a location of the user within the AR meeting space;

determining a set of digital canvases in the AR meeting space, wherein a digital canvas in the set of digital canvases displays digital objects in the AR meeting space and wherein the digital canvas is associated with three-dimensional position information that describes a location of the digital canvas within the AR meeting space;

assembling the set of users and the set of digital canvases into a local scene recreation,

wherein the local scene recreation places the set of users and the set of digital canvases in appropriate locations in the AR meeting space based on the three-dimensional position information, and

wherein the set of users are represented by avatars in the local scene recreation; and

causing the local scene recreation of the AR meeting space to display on a computing device.