Authoring virtual and augmented reality environments via an XR collaboration application

• US 10,573,081 B2
• Filed: 08/03/2018
• Issued: 02/25/2020
• Est. Priority Date: 08/03/2017
• Status: Active Grant

• Current Assignee: Taqtile, Inc.
• Original Assignee: Taqtile, Inc.
• Inventors: Tomizuka, John
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Imperial, Jed-Justin
• Application Number:
  US16/054,975
• Publication Number:
  US 20190043264A1
• Time in Patent Office: 571 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

This disclosure describes techniques that facilitate generating or modifying a template application that provides an application user with interactive experience within a Virtual Reality (VR) environment, an Augmented Reality (AR) environment, or a combination of both (i.e. MR environment). Moreover, this disclosure describes an XR collaboration system that permits an XR author to generate or modify an XR template that is presented to an application user via XR environment-enabled client device. Specifically, an XR author may use the XR collaboration application to add, change, or remove markers within an XR template along with associated virtual content. Further, the XR author may configure appearance criteria associated with markers to control the criteria and sequential order that markers may appear within an XR environment. Similarly, activation criteria may control the appearance and sequential order of virtual content that is associated with markers within the XR environment.

Claims

1. A computer-implemented method, comprising:

retrieving an XR template from a data repository, the XR template corresponding to a real-world and a virtual world representation of a select environment;

receiving, via a client device, an input to assign a marker to an object depicted within the XR template;

capturing, via the client device, virtual content to be associated with the marker;

associating the virtual content with the marker;

generating an appearance criterion that controls a display of the marker within the XR template;

generating an activation criterion that controls a display of the virtual content that is associated with the marker;

updating the XR template to create an updated XR template, the updated XR template including the marker, the appearance criterion, the activation criterion, and the virtual content;

storing the updated XR template;

generating an asset class that groups the virtual content, the appearance criterion, and the activation criterion that is associated with the marker;

identifying additional objects within an XR environment of the updated XR template that correspond to a same type of object as the object;

assigning the asset class to the additional objects, and

wherein, the updated XR template is configured to include assigning the asset class to the additional objects.

2. The computer-implemented method of claim 1, wherein the updated XR template is further configured to display the virtual content in response to the display of the marker.

3. The computer-implemented method of claim 1, further comprising:

determining that the input correspond to a gesture performed by a user of the client device;

analyzing the gesture by identifying a correlation with stored gestures, and

determining that the gesture corresponds to assigning the marker to the object depicted within the XR template, based at least in part on analysis of the gesture.

4. The computer-implemented method of claim 1, further comprising:

updating the XR template to include the activation criterion, the activation criterion corresponding to an audible command, wherein the updated XR template is configured to present the virtual content associated with the marker, based at least in part on fulfillment of the activation criterion.

5. The computer-implemented method of claim 1, wherein the client device is a first client device, and wherein the appearance criterion corresponds to a user'"'"'s client device being within a predetermined distance of the object associated with the marker, and further comprising:

capturing, from a second client device, a geographic location of the second client device within an XR environment of the updated XR template;

determining that the geographic location of the second client device is within the predetermined distance of the object associated with the marker; and

causing display of the marker within the second client device.

6. The computer-implemented method of claim 1, wherein the client device is a first client device, and further comprising:

capturing, from a second client device, a gesture performed by a user of the second client device;

analyzing the gesture by identifying a correlation with stored gestures;

determining that the gesture corresponds to an activation criterion that controls display of the virtual content that is associated with the marker; and

causing display of the virtual content, based at least in part on analysis of the gesture.

7. The computer-implemented method of claim 1, wherein the XR template is associated with a plurality of instructions to maintain serviceability of a physical object, and further comprising:

generating the activation criterion for instances of the virtual content, based at least in part on an order of the plurality of instructions, wherein individual instances of the virtual content correspond to individual instruction to maintain the serviceability of the physical object, and

wherein, the updated XR template further includes the activation criterion for the instances of the virtual content.

8. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

retrieving an XR template from a data repository, the XR template corresponding to a real-world and a virtual world representation of a select environment;

receiving, via a client device, an input to assign a marker to a physical object depicted within the XR template;

assigning, via the client device, virtual content to the marker;

generating an appearance criterion that controls a display of the marker within an XR environment of the XR template;

generating activation criterion that controls a display of the virtual content associated with the marker;

updating the XR template to create an updated XR template, the updated XR template including the marker, the appearance criterion, the activation criterion, and the virtual content;

storing the updated XR template;

generating an asset class that groups the virtual content, the appearance criterion, and the activation criterion that is associated with the marker;

identifying additional physical objects within the XR environment of the updated XR template that correspond to a same type of physical object as the physical object;

assigning the asset class to the additional physical objects, and

wherein, the updated XR template is configured to include assigning the asset class to the additional physical objects.

9. The one or more non-transitory computer-readable media of claim 8, wherein the appearance criterion corresponds to at least one of a threshold temperature, a threshold noise level, a threshold light intensity, a threshold moisture level, or a threshold odor intensity.

10. The one or more non-transitory computer-readable media of claim 8, wherein the activation criterion corresponds to a gesture performed by an application user accessing the updated XR template, the activation criterion further configured to reveal the virtual content, forgo a presentation of the virtual content, or dismiss the display of the marker.

11. The one or more non-transitory computer-readable media of claim 8, wherein the virtual content corresponds to at least one of a video data file, an image file, or an audio file.

12. A system comprising:

one or more processors;

memory coupled to the one or more processors, the memory including one or more modules that are executable by the one or more processors to;

retrieve an XR template from a data repository, the XR template corresponding to a real-world and a virtual world representation of a select environment;

receive, via a client device, an input to assign a marker to a physical object depicted within the XR template;

assign, via the client device, virtual content to the marker;

generate appearance criterion that control a display of the marker associated with the physical object;

generate activation criterion that controls a display of the virtual content associated with the marker;

update the XR template to create an updated XR template, the updated XR template including the marker, the appearance criterion, the activation criterion, and the virtual content;

storing the updated XR template;

generating an asset class that groups the virtual content, the appearance criterion, and the activation criterion that is associated with the marker;

identifying additional physical objects within an XR environment of the updated XR template that correspond to a same type of physical object as the physical object;

assigning the asset class to the additional physical objects, and

wherein, the updated XR template is configured to include assigning the asset class to the additional physical objects.

13. The system of claim 12,

wherein the activation criterion is based at least in part on a predetermined proximity of the client device to the physical object depicted within the updated XR template.

14. The system of claim 12, wherein the marker corresponds to a graphical representation that overlays the physical object depicted within the XR template.