METHODS AND APPARATUS TO DETERMINE OBJECTS TO PRESENT IN VIRTUAL REALITY ENVIRONMENTS

• US 20180011531A1
• Filed: 07/07/2017
• Published: 01/11/2018
• Est. Priority Date: 07/07/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Clement, Manuel Christian, Welker, Stefan
• Granted Patent: US 10,514,752 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

In at least one general aspect, a method can include detecting an interaction with a first object in a virtual reality (VR) environment, producing first contextual information based on user interaction with the first object, determining a second object to display in the VR environment based on the first contextual information, and presenting the second object in the VR environment. The interaction of the user may include at least one of a movement, a speech, or a physiological trait of the user.

Claims

1. A method comprising:

detecting an interaction with a first object in a virtual reality (VR) environment;

producing first contextual information based on user interaction with the first object, the interaction includes at least one of a movement, a speech, or physiological trait of the user;

determining a second object to display in the VR environment based on the first contextual information; and

presenting the second object in the VR environment.

2. The method of claim 1, wherein the interaction is at least two of a movement, a speech, and a physiological trait of the user.

3. The method of claim 1, wherein the movement of the user is obtained via at least one of an eye-tracking device, a head-tracking device, a hand-tracking device, or a body-tracking device.

4. The method of claim 1, wherein the speech of the user is obtained via a microphone.

5. The method of claim 1, wherein the physiological trait of the user is obtained via a biosensor.

6. The method of claim 1, wherein the first contextual information is produced when the user walks toward the first object in the VR environment.

7. The method of claim 1, wherein the first contextual information is produced when the user focuses on the first object in the VR environment.

8. The method of claim 1, wherein producing the first contextual information includes estimating a result to determine whether to display the second object.

9. The method of claim 1, further comprising:

detecting an interaction with a third object in the VR environment;

obtaining second contextual information associated with the interaction with the third object;

determining the second object to display in the VR environment based on the first contextual information and the second contextual information.

10. The method of claim 9, wherein the first contextual information and the second contextual information are weighted when determining the second object based on which of the interaction with the first object or the interaction with the third object occurs first.

11. The method of claim 1, wherein determining the second object includes selecting the second object from a database of objects.

12. The method of claim 1, wherein the first object is a real object.

13. The method of claim 1, wherein the first object is a virtual object.

19. A non-transitory machine-readable media stores machine-readable instructions that, when executed, cause a machine to at least perform the method of claim 1.

14. A method comprising:

detecting an interaction with a first object in a virtual reality (VR) environment;

obtaining first contextual information based on user interaction with the first object;

determining a gaze direction in accordance with the interaction of the user with the first object;

determining a second object to display in the VR environment based on the determined gaze direction; and

presenting the second object in the VR environment.

15. The method of claim 14, wherein the determining the gaze direction includes determining a first intensity level and a second intensity level, wherein:

the first intensity level corresponds to the user focusing on the first object, and

the second intensity level corresponds to the user glancing at the first object.

16. The method of claim 14, wherein when the user is at the first intensity level, the second object is presented to the user.

17. The method of claim 14, wherein when the user is at the second intensity level, the second object is not presented to the user.

18. A system, comprising:

a computing device configured to generate a virtual environment, the computing device including;

a memory storing executable instructions; and

a processor configured to execute the instructions, to cause the computing device to;

detecting an interaction with a first object in a virtual reality (VR) environment;

obtaining first contextual information based on user interaction with the first object, the interaction includes at least one of a movement, a speech, or physiological trait of the user;

determining a second object to display in the VR environment based on the first contextual information; and

presenting the second object in the VR environment.