SYSTEMS AND METHODS FOR A VIRTUAL REALITY EDITOR

• US 20170228130A1
• Filed: 02/09/2017
• Published: 08/10/2017
• Est. Priority Date: 02/09/2016
• Status: Active Grant

• Current Assignee: Unity IPR ApS (Unity Software, Inc.)
• Original Assignee: Unity IPR ApS (Unity Software, Inc.)
• Inventors: Palmaro, Gregory Lionel Xavier Jean
• Granted Patent: US 10,067,636 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0485 Scrolling or panning

Abstract

A system includes processors, a head mounted display, a hand-tracking input device, and an editor engine. The editor engine performs operations including identifying a set of virtual assets, each virtual asset includes data associated with a 3D object, creating a card tray within a virtual environment, creating one or more virtual cards within the card tray, the one or more virtual cards including a first virtual card, the first virtual card is configured with a first interaction mechanic including a triggering gesture and an associated gesture response, the triggering gesture allows the user to interact with the first virtual card within the virtual environment by performing the triggering gesture, detecting performance of the triggering gesture by the user on the first virtual card using the first hand-tracking input device, and based on detecting performance of the triggering gesture, performing the gesture response associated with the first interaction mechanic.

Claims

1. A system comprising:

one or more hardware processors;

a head mounted display (HMD) configured to display a virtual environment to a user wearing the HMD;

a first hand-tracking input device configured to track hand motion of a first hand of the user; and

an editor engine, executable by the one or more hardware processors, configured to perform operations comprising;

identifying a set of virtual assets, each virtual asset includes data associated with a 3D object;

creating a card tray within the virtual environment, the card tray appearing within arms reach of the user;

creating one or more virtual cards within the card tray, the one or more virtual cards including a first virtual card, the first virtual card is configured with a first interaction mechanic, the first interaction mechanic includes a triggering gesture and an associated gesture response, the triggering gesture allows the user to interact with the first virtual card within the virtual environment by performing the triggering gesture on the first virtual card;

detecting performance of the triggering gesture by the user on the first virtual card using the first hand-tracking input device; and

based on detecting performance of the triggering gesture, performing the gesture response associated with the first interaction mechanic.

2. The system of claim 1, wherein each virtual card of the one or more virtual cards represents one or more virtual assets of the set of virtual assets, wherein the first virtual card is a category card associated with one or more virtual assets within an asset category, wherein the first interaction mechanic is a card expansion, wherein the triggering gesture is pressing the first virtual card down within the virtual environment, wherein the gesture response includes creating one or more additional virtual cards within the card tray, each of the one or more additional virtual cards is associated with a virtual asset of the one or more virtual assets within the asset category.

3. The system of claim 2, wherein the one or more additional virtual cards includes a second virtual card, the second virtual card a second interaction mechanic, the second interaction mechanic includes a second triggering gesture and an associated second gesture response, the second triggering gesture is pressing the second virtual card up within the virtual environment, the second gesture response includes removing the one or more additional virtual cards from the card tray.

4. The system of claim 2, wherein the card tray is configured with a third interaction mechanic, the third interaction mechanic includes a third triggering gesture and a third gesture response, the third triggering gesture is grabbing the card tray within the virtual environment, the third gesture response includes scrolling the one or more virtual cards within the card tray based on hand motion of the user.

5. The system of claim 3, wherein the card tray includes a scroll boundary, the operations further comprising:

detecting, during the third triggering gesture, that an edge virtual card within card tray has reached the scroll boundary; and

performing the second gesture response based on the detecting.

6. The system of claim 1, wherein the first interaction mechanic further identifies a first orientation, the first orientation represents an orientation of the first hand of the user in which the triggering gesture is active, the operations further comprising:

identifying a predetermined angle range associated with a first orientation of the first hand of the user;

detecting a first angle of orientation of the first hand-tracking input device; and

determining that the first angle of orientation is within the predetermined angle range associated with the first orientation,

wherein detecting performance of the triggering gesture is further based on the determining that the first angle of orientation is within the predetermined angle range associated with the first orientation.

7. The system of claim 1, the operations further comprising determining a spatial location of a real world surface near the user, wherein creating the card tray within the virtual environment further includes creating the card tray in the virtual environment within a predetermined distance of the real world surface.

8. The system of claim 1, wherein the first virtual card is associated with a first virtual asset of the set of virtual assets, wherein the triggering gesture is pressing the first virtual card down within the virtual environment, wherein the gesture response includes creating a virtual menu within the virtual environment, the virtual menu includes one or more virtual menu objects configured to alter properties of virtual asset.

9. A computer-implemented method comprising:

identifying a set of virtual assets, each virtual asset includes data associated with a 3D object;

creating a card tray within a virtual environment, the card tray appearing within arms reach of a user, the virtual environment is displayed to the user via a head mounted display (HMD) worn by the user;

creating one or more virtual cards within the card tray, the one or more virtual cards including a first virtual card, the first virtual card is configured with a first interaction mechanic, the first interaction mechanic includes a triggering gesture and an associated gesture response, the triggering gesture allows the user to interact with the first virtual card within the virtual environment by performing the triggering gesture on the first virtual card;

detecting performance of the triggering gesture by the user on the first virtual card using a first hand-tracking input device configured to track hand motion of a first hand of the user; and

based on detecting performance of the triggering gesture, performing the gesture response associated with the first interaction mechanic.

10. The method of claim 9, wherein each virtual card of the one or more virtual cards represents one or more virtual assets of the set of virtual assets, wherein the first virtual card is a category card associated with one or more virtual assets within an asset category, wherein the first interaction mechanic is a card expansion, wherein the triggering gesture is pressing the first virtual card down within the virtual environment, wherein the gesture response includes creating one or more additional virtual cards within the card tray, each of the one or more additional virtual cards is associated with a virtual asset of the one or more virtual assets within the asset category.

11. The method of claim 10, wherein the one or more additional virtual cards includes a second virtual card, the second virtual card a second interaction mechanic, the second interaction mechanic includes a second triggering gesture and an associated second gesture response, the second triggering gesture is pressing the second virtual card up within the virtual environment, the second gesture response includes removing the one or more additional virtual cards from the card tray.

12. The method of claim 10, wherein the card tray is configured with a third interaction mechanic, the third interaction mechanic includes a third triggering gesture and a third gesture response, the third triggering gesture is grabbing the card tray within the virtual environment, the third gesture response includes scrolling the one or more virtual cards within the card tray based on hand motion of the user.

13. The method of claim 12, wherein the card tray includes a scroll boundary, the method further comprising:

detecting, during the third triggering gesture, that an edge virtual card within card tray has reached the scroll boundary; and

performing the second gesture response based on the detecting.

14. The method of claim 9, wherein the first interaction mechanic further identifies a first orientation, the first orientation represents an orientation of the first hand of the user in which the triggering gesture is active, the method further comprising:

identifying a predetermined angle range associated with a first orientation of the first hand of the user;

detecting a first angle of orientation of the first hand-tracking input device; and

determining that the first angle of orientation is within the predetermined angle range associated with the first orientation,

wherein detecting performance of the triggering gesture is further based on the determining that the first angle of orientation is within the predetermined angle range associated with the first orientation.

15. The method of claim 9, further comprising determining a spatial location of a real world surface near the user, wherein creating the card tray within the virtual environment further includes creating the card tray in the virtual environment within a predetermined distance of the real world surface.

16. The method of claim 9, wherein the first virtual card is associated with a first virtual asset of the set of virtual assets, wherein the triggering gesture is pressing the first virtual card down within the virtual environment, wherein the gesture response includes creating a virtual menu within the virtual environment, the virtual menu includes one or more virtual menu objects configured to alter properties of virtual asset.

17. A non-transitory machine-readable medium storing processor-executable instructions which, when executed by a processor, cause the processor to:

identify a set of virtual assets, each virtual asset includes data associated with a 3D object;

create a card tray within a virtual environment, the card tray appearing within arms reach of a user, the virtual environment is displayed to the user via a head mounted display (HMD) worn by the user;

create one or more virtual cards within the card tray, the one or more virtual cards including a first virtual card, the first virtual card is configured with a first interaction mechanic, the first interaction mechanic includes a triggering gesture and an associated gesture response, the triggering gesture allows the user to interact with the first virtual card within the virtual environment by performing the triggering gesture on the first virtual card;

detect performance of the triggering gesture by the user on the first virtual card using a first hand-tracking input device configured to track hand motion of a first hand of the user; and

based on detecting performance of the triggering gesture, perform the gesture response associated with the first interaction mechanic.

18. The machine-readable medium of claim 17, wherein each virtual card of the one or more virtual cards represents one or more virtual assets of the set of virtual assets, wherein the first virtual card is a category card associated with one or more virtual assets within an asset category, wherein the first interaction mechanic is a card expansion, wherein the triggering gesture is pressing the first virtual card down within the virtual environment, wherein the gesture response includes creating one or more additional virtual cards within the card tray, each of the one or more additional virtual cards is associated with a virtual asset of the one or more virtual assets within the asset category.

19. The machine-readable medium of claim 18, wherein the one or more additional virtual cards includes a second virtual card, the second virtual card a second interaction mechanic, the second interaction mechanic includes a second triggering gesture and an associated second gesture response, the second triggering gesture is pressing the second virtual card up within the virtual environment, the second gesture response includes removing the one or more additional virtual cards from the card tray.

20. The machine-readable medium of claim 18, wherein the card tray is configured with a third interaction mechanic, the third interaction mechanic includes a third triggering gesture and a third gesture response, the third triggering gesture is grabbing the card tray within the virtual environment, the third gesture response includes scrolling the one or more virtual cards within the card tray based on hand motion of the user.

21. The machine-readable medium of claim 20, wherein the card tray includes a scroll boundary, the processor-executable instructions further cause the processor to:

detect, during the third triggering gesture, that an edge virtual card within card tray has reached the scroll boundary; and

perform the second gesture response based on the detecting.

22. The machine-readable medium of claim 18, wherein the first interaction mechanic further identifies a first orientation, the first orientation represents an orientation of the first hand of the user in which the triggering gesture is active, the processor-executable instructions further cause the processor to:

identify a predetermined angle range associated with a first orientation of the first hand of the user;

detect a first angle of orientation of the first hand-tracking input device; and

determine that the first angle of orientation is within the predetermined angle range associated with the first orientation,

wherein detecting performance of the triggering gesture is further based on the determining that the first angle of orientation is within the predetermined angle range associated with the first orientation.

23. The machine-readable medium of claim 18, the processor-executable instructions further cause the processor to determine a spatial location of a real world surface near the user, wherein creating the card tray within the virtual environment further includes creating the card tray in the virtual environment within a predetermined distance of the real world surface.

24. The machine-readable medium of claim 18, wherein the first virtual card is associated with a first virtual asset of the set of virtual assets, wherein the triggering gesture is pressing the first virtual card down within the virtual environment, wherein the gesture response includes creating a virtual menu within the virtual environment, the virtual menu includes one or more virtual menu objects configured to alter properties of virtual asset.