SYSTEMS AND METHODS FOR USING HIERARCHICAL RELATIONSHIPS OF DIFFERENT VIRTUAL CONTENT TO DETERMINE SETS OF VIRTUAL CONTENT TO GENERATE AND DISPLAY

• US 20180357826A1
• Filed: 06/05/2018
• Published: 12/13/2018
• Est. Priority Date: 06/10/2017
• Status: Abandoned Application

• Current Assignee: Tsunami VR, Inc.
• Original Assignee: Tsunami VR, Inc.
• Inventors: ROSS, David, BREWER, Beth
• Application Number:
  US16/000,835
• Publication Number:
  US 20180357826A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Using hierarchical relationships of different virtual content to determine sets of virtual content to generate and display on a virtual reality, augmented reality, or other user device. Particular embodiments include systems and methods that determine when a first user operating a first user device selects a first object of an environment, identifying the first object from among a plurality of objects, and determine if a plurality of parts of virtual content are stored in association with the first object. If the plurality of parts of the virtual content are stored in association with the first object, the particular embodiments transmit the plurality of parts of the virtual content to the first user device for display of the plurality of parts of the virtual content by the first user device, determine when the first user operating the first user device selects a first part of the virtual content from among the plurality of parts of the virtual content, and determine if one or more subparts of the first part of the virtual content are stored in association with the first part of the virtual content. If the one or more subparts of the first part of the virtual content are stored in association with the first part of the virtual content, the systems and methods transmit the one or more subparts of the first part of the virtual content to the first user device for display of the one or more subparts of the first part of the virtual content by the first user device.

Claims

1. A method for using hierarchical relationships of different virtual content to determine sets of virtual content to generate and display on a virtual reality, augmented reality, or other user device, the method comprising

determining when a first user operating a first user device selects a first object of an environment;

identifying the first object from among a plurality of objects;

determining if a plurality of parts of virtual content are stored in association with the first object;

if the plurality of parts of the virtual content are stored in association with the first object, transmitting the plurality of parts of the virtual content to the first user device for display of the plurality of parts of the virtual content by the first user device;

determining when the first user operating the first user device selects a first part of the virtual content, from among the plurality of parts of the virtual content;

determining if one or more subparts of the first part of the virtual content are stored in association with the first part of the virtual content; and

if the one or more subparts of the first part of the virtual content are stored in association with the first part of the virtual content, transmitting the one or more subparts of the first part of the virtual content to the first user device for display of the one or more subparts of the first part of the virtual content by the first user device.

2. The method of claim 1, the method comprising:

displaying, on a screen of the first user device, the first part of the virtual content along with other parts of the plurality of parts of the virtual content; and

displaying, on the screen of the first user device, the one or more subparts of the first part of the virtual content.

3. The method of claim 1, wherein the virtual content includes a virtual representation of the first object.

4. The method of claim 1, wherein the plurality of parts of the virtual content include text, images, video, audio, static or animated visual representations of components of the first object, or other visual representation of data associated with the first object.

5. The method of claim 1, wherein the plurality of parts of the virtual content include visual representations of two or more components of the first object, wherein the first part of the virtual content comprises a first visual representation of a first component of the first object, and wherein the one or more subparts include one or more visual representation of one or more subcomponents of the first component.

6. The method of claim 1, wherein the plurality of parts of the virtual content include visual representations of two or more components of the first object, wherein the first part of the virtual content comprises a first visual representation of a first component of the first object, and wherein the one or more subparts include one or more of text, image, video or audio about the first component.

7. The method of claim 1, wherein the environment is a real environment, and wherein the first object is a real object in the real environment.

8. The method of claim 1, wherein the environment is a virtual environment, and wherein the first object is a virtual object in the virtual environment.

9. The method of claim 1, wherein the environment is a real environment, and wherein the first object is a virtual object displayed to appear as if the virtual object is in the real environment.

10. The method of claim 1, wherein determining when the first user operating the first user device selects the first object or the first part comprises:

determining if a gaze of the first user intersects with the first object or the first part for a predefined amount of time; and

if the gaze of the first user intersects with the first object or the first part for the predefined amount of time, determining that the first object or the first part is selected by the first user.

11. The method of claim 1, wherein determining when the first user operating the first user device selects the first object or the first part comprises:

determining if a gesture or audio input by the first user selects the first object or the first part; and

if a determination is made that the audio input by the first user selects the first object or the first part, determining that the first object or the first part is selected by the first user.

12. The method of claim 1, wherein determining when the first user operating the first user device selects the first object or the first part comprises:

determining if input by the first user generated using a glove, a controller or other peripheral device selects the first object or the first part; and

if a determination is made that the input by the first user generated using the glove, the controller or the other peripheral device selects the first object or the first part, determining that the first object or the first part is selected by the first user.

13. The method of claim 1, wherein determining when the first user operating the first user device selects the first object or the first part comprises:

determining if a condition is met; and

if the condition is met, determining that the first object or the first part is selected by the first user.

14. The method of claim 13, wherein the condition is determined to have been met when a position of the first user in the environment is within a predefined distance of a position of the first object or the first part in the environment.

15. The method of claim 13, wherein the condition is determined to have been met when the first object or the first part is inside a field of view of the first user.

16. The method of claim 13, wherein the condition is determined to have been met when the first user moves from an initial position of the first user in the environment towards the first object or the first part.

17. The method of claim 1, wherein the first object is among a plurality of objects in the environment.

18. The method of claim 1, the method comprising:

determining when a second user operating a second user device selects the first object;

transmitting the plurality of parts of the virtual content to the second user device for display of the plurality of parts of the virtual content by the second user device;

determining when the second user operating the second user device selects a second part of the virtual content, from the plurality of parts of the virtual content;

determining if one or more subparts of the second part of the virtual content are stored in association with the second part of the virtual content; and

if the one or more subparts of the second part of the virtual content are stored in association with the second part of the virtual content, transmitting the subparts of the second part of the virtual content to the second user device for display of the subparts of the second part of the virtual content by the second user device.

19. The method of claim 1, the method comprising:

determining when the first user operating the first user device selects a second part of the virtual content, from the plurality of parts of the virtual content;

determining if one or more subparts of the second part of the virtual content are stored in association with the second part of the virtual content; and

if the one or more subparts of the second part of the virtual content are stored in association with the second part of the virtual content, transmitting the subparts of the second part of the virtual content to the first user for display of the subparts of the second part of the virtual content by the first user device.

20. One or more non-transitory machine-readable media embodying program instructions that, when executed by one or more machines, cause the one or more machines to implement the method of claim 1.